Predicting the Future of Predictive Modeling

It is now well established that predictive models can be generated from the artifacts of software projects. So it is time to ask “what’s next?”. I suggest that predictive modeling tools can and should be refactored to address the near-term issue of decision systems and the long-term goal of social reasoning. INTRODUCTION: In software engineering (SE) there is much we are seeing, but little we are learning. The sheer volume of SE data is overwhelming. As of October 2011, 10,000 projects are monitored at http://CIA.vcwith one new commit every 17 seconds. The open source platform SourceForge.Net hosts over 300K projects, and according to Github.com 1M people host 2.9M GIT repositories. The bug database of the Mozilla Firefox projects now contains almost 700K reports according to Ohloh.Net. Yet from that data, we have extracted nearly zero general principlesthe usual result is, across all this data, is that what works one project may not work on another [28, 29]. Hence there is an urgent need for better analysis of this data. Further, due to the volume of information, that analysis must be (at least partially) automated. Hence, AI research in data mining has been widely adopted in predictive modeling community with SE. In this brief note, I critique the state of the art in that field and suggest a future direction. In summary, I think we need to move beyond mere predictive modeling. Last century, it was not known if software projects contained sufficient structure to support data mining, though some preliminary results by Porter were encouraging [37]. Now, we know better. Many different kinds of artifacts from software projects contain a signal that can be revealed via data mining including: • apps store data [13]; • process data which can predict overall project effort [18]; • process models which can find effective project changes [30, 40]; • operating system logs that predict software power consumption [15]; • natural language requirements documents which can be text-mined to find connections between program components [14]; • XML descriptions of design patterns that can be used to recommend particular designs [35]; • email lists that reveal the human networks inside software teams [2]; • execution traces that generate normal interface usage patterns [10]; • bug databases that can generate defect predictors to guide inspection teams to where the code is most likely to fail [23, 27, 34, 37]. It is now well-established that predictive models can be built from software projects artifacts. So it is now time to ask “what’s next?”. Stepney et al. [43] assert that an ideal research roadmap “decomposes into identified intermediate research goals, whose achievement brings scientific or economic benefit, even if the project as a whole fails”. Hence I propose the following progression that can refocus our exist tools and talent into new and novel areas. According this progression, we are now leaving the age of prediction systems and entering the age of decision systems. After that, we should move to the era of social reasoning: